1. Technical Field
This invention relates to the field of computer software systems and more specifically to a method for allowing representation of a software application's window hierarchy in an efficient internal format for a speech navigator.
2. Description of the Related Art
In recent years, various software systems have been developed to enable an application program running on a computer to recognize and respond to voice commands. Such programs are advantageously designed as independent or stand-alone systems which are designed to provide voice recognition capabilities to existing commercially available target application programs. Thus, sophisticated voice recognition capability can be economically made available for a wide variety of commercial application software, without modifying the existing source code of such application software.
Voice recognition systems are designed to allow user data to be entered in a target application program by means of spoken words (e.g. dictation of a report in a word processing application program). In addition, some systems also enable such target application programs to respond to voice commands for controlling the software (e.g., opening and closing windows, choosing program options, and causing the application software to perform certain functions). Systems which allow voice control of a target application program are sometimes called voice navigators. Significantly, the design of an independently developed voice navigator system which is capable of associating voice commands with equivalent keyboard or mouse actuated control functions for a wide variety of commercially available application programs, has been hindered by certain difficulties.
Conventional voice navigation programs are typically designed to dynamically analyze an active window in order to determine the attributes of the various screen objects and determine a command vocabulary set for controlling such objects. In order to perform this dynamic analysis, there are several features of every window in a target application that the speech navigator can probe to determine the attributes of a particular object. These features include the (1) window class name, (2) window text, and (3) window identification number. The window class name indicates the type of the object (e.g., "BUTTON", "LISTBOX", "EDIT BOX", or "SCROLLBAR"). The window text feature is specific text associated with a window which allows a application program user to understand the function or relevance of a particular window. Conventional navigators will determine how to use the window text based upon the class name. For example, if the class name is "BUTTON" the window text of the button would be the words which would normally appear on the face of the button. Accordingly, the navigator would use the window text to determine the spoken command which can be used to activate the button. In other words, by probing the target application program regarding the window text, the navigator can associate certain spoken text to a particular button or control. Examples of window text might include words such as "OK" or "CANCEL" in the case of a push-button, or a list of items in the case of a list box. Finally, the navigator may also probe the application program for the window identification number as a way to internally distinguish controls which may otherwise look similar. The window identification number uniquely identifies a child window from other child windows having the same parent window.
Significantly, certain types of controls, such as "BUTTON", "LISTBOX", "EDIT BOX", or "SCROLLBAR" controls, are generic within the Windows operating systems, and may easily be recognized by the navigator when it dynamically probes a target application. However, when a target program developer has chosen to create certain custom controls in a particular application, such custom controls cannot share a generic class name such as those listed above. Consequently, navigator programs which rely upon such class names to distinguish these object types will not recognize the object as any known control, and will not therefore respond to any voice commands directed to such controls.
Further, conventional voice navigation programs base an active set of available voice commands on the currently active window and active control of the target program. For programs implemented in a Windows environment, this generally means that a computer user has access only to those voice actuated commands which are specifically associated with the particular window which has most recently been active or has been designated as the foreground window. Thus, if a user should wish to make use of a target application's command, button or other control that is not associated with a window which presently is considered to be the foreground window, it is necessary to first use a voice command to change the system status to cause a different window to be designated the foreground window. Only then will the voice command be recognized by the voice navigator system.
Further, because prior art voice navigators dynamically analyze the target application program to determine the specific words or phrases will actuate a particular window or control, they are generally constrained to allow only a single command or name for actuation or operation of a given control. This limits the flexibility of the voice navigator to the extent that it will not permit two or more commands from being available to operate a single control. Such prior art voice navigators are also limited in their ability to recognize custom user interface controls, such as buttons or list boxes, which may have been uniquely developed for use with a particular application. Although such custom controls may operate in a manner which is identical to standard controls, the custom controls will not be recognized by the navigator as such, unless the source code of the navigator is specifically modified to do so.
Finally, it has been found that certain types of menus and tool bars which may appear in a particular window of a target application, will never be treated as a foreground window, in an operating system sense, which would permit conventional voice navigators to recognize voice commands directed to such controls. This is a significant problem, as it will result in the voice navigator never being able to control the particular menu or tool bar.
Thus, it would be desirable to provide a method of representing the window object hierarchy and attributes internally, within a voice navigator application. In addition, it would be desirable to provide a method for internally representing a target application program to a voice navigator in a manner which would permit the voice navigator to control custom design controls of the target application. It would also be desirable to provide a method for internally representing a target application program to a voice navigator so as to permit the voice navigator to use controls in windows which are not presently designated as foreground windows. Finally, it would be desirable to provide a method for internally representing a window hierarchy of a target application program, to a voice navigator, without any recompiling of the source code of either the target application or the voice navigator.